Logical Markov Decision Programs

Motivated by the interest in relational reinforcement learning, we introduce a novel representation formalism, called logical Markov decision programs (LOMDPs), that integrates Markov Decision Processes with Logic Programs. Using LOMDPs one can compactly and declaratively represent complex relational Markov decision processes. Within this framework we then develop a theory of reinforcement learning in which abstraction (of states and actions) plays a major role. Various convergence results are presented, as well as some experiments that validate the approach. The theory presented should provide a sound basis for further developments in relational reinforcement learning.